Can ChatGPT help patients answer their otolaryngology questions?

Background: Over the past year, the world has been captivated by the potential of artificial intelligence (AI). The appetite for AI in science, specifically healthcare is huge. It is imperative to understand the credibility of large language models in assisting the public in medical queries. Objective: To evaluate the ability of ChatGPT to provide reasonably accurate answers to public queries within the domain of Otolaryngology. Methods: Two board-certified otolaryngologists (HZ, RS) inputted 30 text-based patient queries into the ChatGPT-3.5 model. ChatGPT responses were rated by physicians on a scale (accurate, partially accurate, incorrect), while a similar 3-point scale involving confidence was given to layperson reviewers. Demographic data involving gender and education level was recorded for the public reviewers. Inter-rater agreement percentage was based on binomial distribution for calculating the 95% confidence intervals and performing significance tests. Statistical significance was defined as p < .05 for two-sided tests. Results: In testing patient queries, both Otolaryngology physicians found that ChatGPT answered 98.3% of questions correctly, but only 79.8% (range 51.7%-100%) of patients were confident that the AI model was accurate in its responses (corrected agreement = 0.682; p < .001). Among the layperson responses, the corrected coefficient was of moderate agreement (0.571; p < .001). No correlation was noted among age, gender, or education level for the layperson responses. Conclusion: ChatGPT is highly accurate in responding to questions posed by the public with regards to Otolaryngology from a physician standpoint. Public reviewers were not fully confident in believing the AI model, with subjective concerns related to less trust in AI answers compared to physician explanation. Larger evaluations with a representative public sample and broader medical questions should immediately be conducted by appropriate organizations, governing bodies, and/or governmental agencies to instill public confidence in AI and ChatGPT as a medical resource. Level of Evidence: 4.

Pediatric Airway Balloon Dilation: Urge for Safety Redesign.

Airway balloons are widely used by otolaryngologists to treat laryngotracheal stenosis. We review an adverse event and interventions that may prevent similar occurrences. There are no other reports of similar incidents in the Food and Drug Administration Manufacturer and User Facility Device Experience. We implore Acclarent to redesign their airway balloon device to prevent accidental airway irrigation.

Evaluating the Current Ability of ChatGPT to Assist in Professional Otolaryngology Education.

Objective: To quantify ChatGPT's concordance with expert Otolaryngologists when posed with high-level questions that require blending rote memorization and critical thinking. Study Design: Cross-sectional survey. Setting: OpenAI's ChatGPT-3.5 Platform. Methods: Two board-certified otolaryngologists (HZ, RS) input 2 sets of 30 text-based questions (open-ended and single-answer multiple-choice) into the ChatGPT-3.5 model. Responses were rated on a scale (correct, partially correct, incorrect) by each Otolaryngologist working simultaneously with the AI model. Interrater agreement percentage was based on binomial distribution for calculating the 95% confidence intervals and performing significance tests. Statistical significance was defined as P < .05 for 2-sided tests. Results: In testing open-ended questions, the ChatGPT model had 56.7% of initially answering questions with complete accuracy, and 86.7% chance of answer with some accuracy (corrected agreement = 80.1%; P < .001). For repeat questions, ChatGPT improved to 73.3% with complete accuracy and 96.7% with some accuracy (corrected agreement = 88.8%; P < .001). For multiple-choice questions, the ChatGPT model performed substantially worse (43.3% correct). Conclusion: ChatGPT currently does not provide reliably accurate responses to sophisticated questions in Otolaryngology. Professional societies must be aware of the potential of this tool and prevent unscrupulous use during test-taking situations and consider guidelines for clinical scenarios. Expert clinical oversight is still necessary for myriad use cases (eg, hallucination).

Ensuring tympanostomy tube follow-up during the COVID-19 pandemic.

OBJECTIVES: The impact of the COVID-19 pandemic on surgical practice was widespread. Local health restrictions in many communities limited the number and types of patients seen and treated. Our goal is to understand the discrepancies in follow-up for bilateral myringotomy with tubes (BMT) and understand whether the pandemic made it more difficult for certain patient populations to continue routine follow up. METHODS: This cross-sectional study abstracted demographic and clinical data from the medical charts of randomized pediatric patients who underwent a BMT procedure between 2017 and 2022 at a tertiary pediatric academic care center site. Suboptimal follow-up was defined as no clinic visits within the first 6 months of surgery, and less than 3 follow-ups within a 24-month period after surgery. Data regarding secondary surgeries, clinical variables, race, zip code, and dates and number follow-ups were recorded from randomly chosen eligible charts. Complications were defined as need for revision surgery or development of post-tympanostomy sequalae such as perforation, cholesteatoma, or granuloma. RESULTS: Pre-pandemic (N = 150) patients from 2017 to 2019 and post-pandemic (N = 150) patients from 2020 to 2022 cohorts were analyzed. No significant differences were identified between the groups regarding age, presence of combined surgery, anesthesia class, diagnosis, or race. There was no significant difference in rate of complications or need for revision surgery between either cohort nor when stratified for demographics. CONCLUSION: Limitations imposed upon medical care including outpatient clinics due COVID restrictions did not have an impact on tympanostomy tube follow-up regarding frequency of visits and risk of complication or revision surgery. LEVEL OF EVIDENCE: IV.

A Customized Triggers Program: A Children's Hospital's Experience in Improving Trigger Usability.

BACKGROUND: Despite the growth of patient safety programs across the United States, errors and adverse events remain a source of patient harm. Many hospitals rely on retrospective voluntary reporting systems; however, there are opportunities to improve patient safety using novel tools like trigger programs. METHODS: Children's National Hospital developed a unique pediatric triggers program that offers customized, near real-time reports of potential safety events. Our team defined a measure to quantify clinical utility of triggers, termed "trigger signal," as the percentage of cases that represent true adverse or near-miss events (numerator) per total triggers activated (denominator). Our key driver diagram focused on unifying the program structure, increasing data analytics, promoting organizational awareness, and supporting multidisciplinary end user engagement. Using the model for improvement, we aimed to double overall trigger signal from 8% to 16% and sustain for 12 months. RESULTS: The trigger signal increased from 8% to 41% and sustained during the coronavirus disease 2019 pandemic. A balancing measure of time to implement a new trigger decreased. Key interventions to increase trigger signal were change in the program structure, increasing stakeholder engagement, and development of self-service reports for end users. CONCLUSIONS: Children's National Hospital's triggers program highlights successful evolution of an iterative, customized approach to increase clinical utility that hospitals can implement to impact real-time patient care. This triggers program requires an iterative, customized approach rather than a "1-size-fits-all," static paradigm to add a new dimension to current patient safety programs.

Slips, Trips, and Falls: A Quality Improvement Initiative.

Employee safety and the reduction of Days Away, Restricted, or Transferred are a focus of the Solutions for Patient Safety Network. One significant contributor to the Days Away, Restricted, or Transferred rate at Children's National Hospital is employee slips, trips, or falls. Methods: Children's National Hospital implemented a multidisciplinary quality improvement with executive leadership vision and support. We implemented quality techniques (including Key Driver Diagrams, Pareto Charts, and continuous Plan-Do-Study-Act) and designed novel Environmental Services interventions. Results: Children's National Hospital achieved a 44.3% reduction in monthly average reported slips, trips, or fall events from baseline and sustained over a 2-year study period. Conclusion: A leadership-driven multidisciplinary approach to quality initiatives with team leaders capable of making and enacting real-time policy changes led to novel interventions and a successful reduction of employee slips, trips, and falls events over time, which are broadly generalizable.

COVID-19 Patient Vaccine Program Design and Implementation: An Academic Children's Hospital's Model, Approach, and Outcomes.

To slow the spread of the 2019 novel coronavirus disease (COVID-19) and reduce the associated morbidity and mortality, the Children's National Hospital developed a multidisciplinary, collaborative vaccine program aimed at equitably and expeditiously vaccinating the pediatric population of the surrounding community. Interdepartmental collaboration, professional expertise, and community partnerships allowed for a dynamic and successful program design that began as large volume-centralized vaccine clinics and expanded to smaller volume ambulatory clinics. This strategy proved successful at meeting local vaccine demand; however, strategies to improve vaccine uptake in communities with high rates of hesitancy are still needed to maximize vaccine equity.

Making the Business Case for Quality and Safety.

There is broad understanding and appreciation that quality and safety are indispensable parts of the business enterprise of delivering care. However, because health care organizations have resource constraints and competing priorities, leaders and managers must create, demonstrate, and articulate a business case for continuing to prioritize investments in quality and safety. To accomplish this, one must leverage financial principles with compelling story-telling. Success creates a virtuous cycle whereby ongoing investments in robust structures increase returns (value defined as improvements in quality and safety outcomes), and cost savings are reinvested to continue to improve delivery of high-quality care.

Common seasonal respiratory viral infections in children before and during the coronavirus disease 2019 (COVID-19) pandemic.

OBJECTIVES: To describe the incidence of seasonal respiratory viral infections (s-RVIs) before and during the coronavirus disease 2019 (COVID-19) pandemic and to compare virus-specific patient outcomes in pediatric patients. DESIGN: A retrospective cross-sectional study including patient admissions to the Children's National Hospital between October 1, 2015, and December 31, 2020. RESULTS: Among 12,451 patient admissions between March 15 and December 31, 2020 (cohort 1), 8,162 (66%) were tested for severe acute respiratory coronavirus virus 2 (SARS-CoV-2), and 249 (2.0%) were positive. Among 10,986 patient admissions between April 1 and December 31, 2020 (cohort 2), 844 (8%) were tested for s-RV upon admission and 160 were positive. Thus, 1.5% of patient admissions were associated with laboratory-confirmed s-RVIs. Among the 49,901 patient admissions during a viral season between October 1, 2015, and March 31, 2020 (cohort 3), 7,539 (15%) were tested for s-RV upon admission and 4,531 were positive; thus, 9.0% of patient admissions were associated with laboratory-confirmed s-RVIs. hHRV/rENT was the most detected virus, but the detection rate decreased substantially (31% vs 18%; P < .001) during the COVID-19 pandemic. No patients had RSV, influenza, hMPV, hPIV, or hCoV detected upon admission after April 21, 2020. The 3 patient cohorts had no statistically significant difference in the percentage of ICU admissions (10.8% vs 15.0% vs 14.2%; P > .05) or death at discharge (0.8% vs 0.6% vs 0.5%; P > .05). CONCLUSIONS: Compared to COVID-19, s-RVI cases were associated with a higher proportion of inpatient admissions but were similar in ICU admission and death rates in hospitalized pediatric patients. Public health interventions for preventing COVID-19 were highly effective in preventing pediatrics s-RVIs.

Oral Intubation Attempts in Patients With a Laryngectomy: A Significant Safety Threat.

It is impossible to secure the airway of a patient with "neck-only" breathing transorally or transnasally. Surgical removal of the larynx (laryngectomy) or tracheal rerouting (tracheoesophageal diversion or laryngotracheal separation) creates anatomic discontinuity. Misguided attempts at oral intubation of neck breathers may cause hypoxic brain injury or death. We present national data from the American Academy of Otolaryngology-Head and Neck Surgery, the American Head and Neck Society, and the United Kingdom's National Reporting and Learning Service. Over half of US otolaryngologist respondents reported instances of attempted oral intubations among patients with laryngectomy, with a mortality rate of 26%. UK audits similarly revealed numerous resuscitation efforts where misunderstanding of neck breather status led to harm or death. Such data underscore the critical importance of staff education, patient engagement, effective signage, and systems-based best practices to reliably clarify neck breather status and provide necessary resources for safe patient airway management.

Reducing Employee Injury Rates with a Hospital-wide Employee Safety Program.

Introduction: Despite the well-known dangers of working in the healthcare industry, healthcare organizations have historically accepted workplace injuries as business as usual. In 2017, Children's National Hospital began our Employee and Staff Safety program to drive down the employee injury rate and address this disturbing industry trend. Methods: With guidance and support from executive leadership, we created an Employee and Staff Safety program that aligned employee safety work with existing patient safety and quality improvement efforts. Team leads collected and analyzed baseline employee injury data and identified areas of highest injuries. Dedicated subcommittees focused on five specific areas: slips, trips, and falls; sharps injuries; blood and body fluid exposures; verbal and physical violence; and overexertion injuries. Subcommittees established aims, identified key drivers, and brainstormed interventions for tests of change. Results: Because the inception of the Employee and Staff Safety program, Children's National has seen significant reductions in our Days Away Restricted or Transfer (DART) rate. The DART rate shows a sustained 37% reduction since the baseline period of FY16-FY17 (1.48 injuries/200,000 h worked to 0.93 injuries/200,000 h worked). The regression trend shows a significant decrease (38.3%) in DART injuries, from 1.544 to 0.952 over 56 months; P = 0.016. Conclusions: Active leadership support and analyzing data on specific employee harm areas coupled with targeted interventions, helped improve Children's National's DART rate. The Employee and Staff Safety program's success in utilizing patient safety and quality improvement tools creates a generalizable framework for other hospitals to advance their high-reliability journey.

Comparison of Clinical Features of COVID-19 vs Seasonal Influenza A and B in US Children.

Importance: Compared with seasonal influenza, the clinical features and epidemiologic characteristics of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) and coronavirus 2019 (COVID-19) in US children remain largely unknown. Objective: To describe the similarities and differences in clinical features between COVID-19 and seasonal influenza in US children. Design, Setting, and Participants: This retrospective cohort study included children who were diagnosed with laboratory-confirmed COVID-19 between March 25 and May 15, 2020, and children diagnosed with seasonal influenza between October 1, 2019, and June 6, 2020, at Children's National Hospital in the District of Columbia. Exposures: COVID-19 or influenza A or B. Main Outcomes and Measures: Rates of hospitalization, admission to the intensive care unit, and mechanical ventilator use and the association between underlying medical conditions, clinical symptoms, and COVID-19 vs seasonal influenza. Results: The study included 315 patients diagnosed with COVID-19 (164 [52%] male; median age, 8.3 years [range, 0.03-35.6 years]) and 1402 patients diagnosed with seasonal influenza (743 [53%] male; median age, 3.9 years [range, 0.04-40.4 years]). Patients with COVID-19 and those with seasonal influenza had a similar hospitalization rate (54 [17%] vs 291 [21%], P = .15), intensive care unit admission rate (18 [6%] vs 98 [7%], P = .42), and use of mechanical ventilators (10 [3%] vs 27 [2%], P = .17). More patients hospitalized with COVID-19 than with seasonal influenza reported fever (41 [76%] vs 159 [55%], P = .005), diarrhea or vomiting (14 [26%] vs 36 [12%], P = .01), headache (6 [11%] vs 9 [3%], P = .01), body ache or myalgia (12 [22%] vs 20 [7%], P = .001), and chest pain (6 [11%] vs 9 [3%], P = .01). Differences between patients hospitalized with COVID-19 vs influenza who reported cough (24 [48%] vs 90 [31%], P = .05) and shortness of breath (16 [30%] vs 59 [20%], P = .13) were not statistically significant. Conclusions and Relevance: In this cohort study of US children with COVID-19 or seasonal influenza, there was no difference in hospitalization rates, intensive care unit admission rates, and mechanical ventilator use between the 2 groups. More patients hospitalized with COVID-19 than with seasonal influenza reported clinical symptoms at the time of diagnosis.

Reducing Vancomycin Use in a Level IV NICU.

BACKGROUND AND OBJECTIVES: Vancomycin remains one of the most commonly prescribed antibiotics in NICUs despite recommendations to limit its use for known resistant infections. Baseline data revealing substantially higher vancomycin use in our NICU compared to peer institutions informed our quality improvement initiative. Our aim was to reduce the vancomycin prescribing rate in neonates hospitalized in our NICU by 50% within 1 year and sustain for 1 year. METHODS: In the 60-bed level IV NICU of an academic referral center, we used a quality improvement framework to develop key drivers and interventions including (1) physician education with benchmarking antibiotic prescribing rates; (2) pharmacy-initiated 48-hour antibiotic time-outs on rounds; (3) development of clinical pathways to standardize empirical antibiotic choices for early-onset sepsis, late-onset sepsis, and necrotizing enterocolitis; coupled with (4) daily prospective audit with feedback from the antimicrobial stewardship program. RESULTS: We used statistical process u-charts to show vancomycin use declined from 112 to 38 days of therapy per 1000 patient-days. After education, pharmacy-initiated 48-hour time-outs, and development of clinical pathways, vancomycin use declined by 29%, and by an additional 52% after implementation of prospective audit with feedback. Vancomycin-associated acute kidney injury also declined from 1.4 to 0.1 events per 1000 patient-days. CONCLUSIONS: Through a sequential implementation approach of education, standardization of care with clinical pathways, pharmacist-initiated 48-hour time-outs, and prospective audit with feedback, vancomycin days of therapy declined by 66% over a 1-year period and has been sustained for 1 year.

Apparent Cause Analysis: A Safety Tool.

Causal analysis is a core function of safety programs. Although established protocols exist for conducting root cause analysis for serious safety events, there is limited guidance for apparent cause analysis (ACA) in health care. At our institution, through a novel facilitated ACA approach, we aim to improve safety culture and provide a clear approach to address precursor safety events and near-miss safety events. We define facilitated ACA as limited investigation (scope and duration) of a safety event that resulted in little to no harm. These investigations require fewer resources and focus on preventive strategies. Our facilitated ACA model, with an operational algorithm and structured process, was developed and implemented at our tertiary-care, freestanding, urban pediatric hospital in 2018. Sixty-four ACAs were completed, and 83% were identified with the algorithm. Process measures, including time from event reporting to ACA launch (median 3 days; interquartile range 2-6 days), are tracked. Patient safety consultants averaged 5 hours to complete a facilitated ACA. A median of 3 disciplines or departments participated in each facilitated ACA. Through an iterative process, we implemented a structured process for facilitated ACA, and the model's strength includes (1) right event, (2) right team, (3) right analysis, and (4) right action plans. This novel facilitated ACA model may support organizational cause analysis and improve safety culture with higher-reliability processes.

Perioperative Harm Index facilitates prioritization of improvement initiatives.

INTRODUCTION: Perioperative services constitute a significant portion of the care delivery, the impact, and the potential risk in healthcare organizations. Tremendous attention has been paid towards hospital-acquired conditions; however perioperative services have not received similar attention. There is a need for a standardized manner to report on conditions in perioperative services which facilitates prioritization of quality improvement initiatives. MATERIALS AND METHODS: Preventable harm and quality of care indicators were selected based on a review of the literature and available datasets, as well as from safety and quality measures in our organization. Metrics were derived from myriad national quality improvement initiatives and collaboratives. A structure was created to obtain the metrics in a near real-time manner and present the Perioperative Harm Index across the organization. Specific initiatives were targeted as necessitating immediate, short-term, or longer duration prioritization for improvement initiatives. RESULTS: A Perioperative Harm Index was created using 11 metrics that represent the spectrum of surgical care. The metrics facilitate prioritization of improvement initiatives and have resulted in improvement projects including perioperative normothermia in neonatal intensive care unit patients having procedures in the operating room, reduction of post-operative nausea and vomiting, and decrease in surgical site infections in selected procedures. CONCLUSIONS: A Perioperative Harm Index facilitates immediate shared understanding of the harm resulting from the care of surgical patients. As such, this index enables rapid and rationale prioritization for improvement activities. Our harm index is shared, is broadly generalizable, and has facilitated prioritization of improvement opportunities and appropriate allocation of improvement resources at our organization. LEVELS OF EVIDENCE: Level V.

Tracheostomy manipulations: Impact on tracheostomy safety.

IMPORTANCE: Tracheotomy is one of the riskiest procedures for composite morbidity within pediatric otolaryngology. During the postoperative period, each time the tracheostomy tube is manipulated, there is opportunity for morbidity (e.g. a patient is vulnerable to accidental decannulation and airway loss). OBJECTIVE: To identify areas of improvement in caring for "fresh tracheostomy" patients by determining the number of times a tracheostomy tube is manipulated from placement until discharge. The hypothesis is that the more a tracheostomy is manipulated, the higher probability of morbidity. METHODS: A quality improvement initiative was conducted to map the care of patients who underwent tracheostomy placement over 12 months. Tracheostomy care and manipulation by all providers were reviewed. Complications, wound care, and respiratory treatments were also evaluated. RESULTS: Patients were hospitalized for an average of 39 days (7-140) following tracheostomy. The first tracheostomy tube change occurred on average 6 days (5-10) following placement. Tracheostomy tubes were manipulated an average of 6 (2.5-11.9) times a day to amount to 216 (51-1091) times between placement and discharge. Bedside nurses and respiratory therapists were responsible for 95% of these actions; physicians accounted for 4%. There were 6 tracheostomy related complications. Three were accidental decannulations resulting in cardiopulmonary arrest. One of these caused long term patient morbidity. Patients with more than 4 manipulations per day during the 2 weeks following tracheostomy tube placement, were more likely to have a tracheostomy related complication than those with less than 4 (OR: 12.5; 95% CI: 1.2-130.6; P = 0.0349). INTERPRETATION: While uncommon, complications related to tracheostomy can have serious long term effects and at best prolongs length of stay for patients. Reducing the number of tracheostomy manipulations may provide safer postoperative care ultimately reducing morbidity and potentially mortality; children on average have 6 tracheotomy manipulations/day with only 2% being by the physician.

10,000 Good Catches: Increasing Safety Event Reporting In A Pediatric Health Care System.

BACKGROUND: In 2014, Children's National Health System's executive leadership team challenged the organization to double the number of voluntary safety event reports submitted over a 3-year period; the intent was to increase reliability and promote our safety culture by hardwiring employee event reporting. METHODS: Following a Donabedian quality improvement framework of structure, process, and outcomes, a multidisciplinary team was formed and areas for improvement were identified. The multidisciplinary team focused on 3 major areas: the perceived ease of reporting (ie, how difficult is it to report an event?); the perceived safety of reporting (ie, will I get in trouble for reporting?); and the perceived impact of reporting (ie, does my report make a difference?) technology, making it safe to report, and how reporting makes a difference. The team developed a key driver diagram and implemented interventions designed to impact the key drivers and to increase reporting. RESULTS: Children's National increased the number of safety event reports from 4,668 in fiscal year 2014 to 10,971 safety event reports in fiscal year 2017. Median event report submission time was decreased by nearly 30%, anonymous reporting decreased by 69%, the number of submitting departments increased by 94%, and the number of reports submitted as "other" decreased from a baseline of 6% to 2%. CONCLUSIONS: Children's National Health System's focus on increasing safety event reporting resulted in increased organizational engagement and attention. This initiative served as a tangible step to improve organizational reliability and the culture of safety and is readily generalizable to other hospitals.